It is known in the prior art that Electromagnetic (EM) radiation can be characterized mathematically as waves as well as photons. At long wavelengths, e.g. radio frequencies, the photons have little energy and the wave properties predominate. At shorter wavelengths, e.g. the visible light bands, photons have considerable energy and are used in designing conventional light detecting systems. However, because designing conventional electronic imaging depends on photon detectors, such as CCD and CMOS devices, the apparatus cannot directly detect the phase of EM waves in the light bands. Not being able to detect and process phase means EM information is lost. An explanation of the difficulties of capturing EM phase with prior art intensity detectors is in An Introduction to Imaging, Milton Keynes: The Open University, 1992, Units 1 & 2, pp. 78ff, ISBN: 0-7492-5051-8.
The mathematics, describing how EM waves create images using conventional lenses follows Huygens' and Fermat's Principles, as expanded by Fourier, is found in: Parrent, GB, “The New Physical Optics Notebook,” The Society of Photo-Optical Instrumentation Engineers, 1989, chaps. 1-4, ISBN: 0-8194-0130-7; Lipson, S G, et. al., “Optical Physics,” 3d Ed., Cambridge, 1995, sects. 2.7 and 4.4ff; ISBN: 0-521-43631-1; and, Ditchburn, R W, Light, Dover, 1991, sect. 3.8, ISBN: 0-486-66667-0.
The technical background of the present invention will be more readily understood by reference to the following detailed description, taken with reference to the accompanying drawings, in which FIGS. 1, 2A and 2B (prior art) illustrate Huygens' Principle which describes how refraction and diffraction phenomena are used to reconstruct an image from an irradiated scene in terms of wave fronts convolving on an image plane; FIG. 3 illustrates how a light field diffracted via an aperture mask creates a phase map with its interference maxima planes; and, FIGS. 4, 5, and 6 illustrate, as applied in various embodiments of the present invention, how wave fronts diffracted via an aperture mask interfere creating a point spread function (PSF), an Airy disk, or a Fraunhofer function on a target plane (viz: Ditchburn, Chap. VI and FIG. 6.5; and Taylor, C A, Images, London: Wykeham, 1978, sects. 2.4 and 3, FIG. 3.8, ISBN: 0-85109-620-4.)
A lens organizes light waves to assemble an image on the image plane using refractive principles; these principles depend directly on superposition of the waves at the image plane. For example, one can project an image of a window in a room onto a wall opposite the window by placing the lens at its focal point from the wall. This demonstration can be duplicated by moving the lens to any location in the room and correspondingly moving the image plane to its new focal length because the wave fronts of all light waves emanating from the window scene exist in every part of the room, not just on the wall. Refraction therefore is actually the re-arrangement of light waves by a lens to produce an image on the image plane.
This is illustrated in FIG. 1 (prior art) which indicates how the lens delays each light ray, acting as a “phase-adjuster” according to the lens' geometry, so that time taken for each wave front (or wavelet) to arrive from a point on the object at the image plane is identical, creating a one-to-one correspondence map of constructive “interference maximums” points on the image plane representing the object or scene (FIG. 1 herein is overlaid from FIG. 3.10 in Taylor, op. cit., pp. 67-69).
At the points on the image plane where the EM rays' phases coincide the intensity (defined in the art as the square of the amplitude time-averaged over the wave length) is at a maximum, while “everywhere else the phase relationships are so random that the waves effectively cancel each other out and we have interference minima” (Taylor, op. cit. p. 68). This is shown by following the ray tracings in FIG. 1, which illustrates Huygens' Principle. From point P, 51, on the object scene 52 (letter “A”) to point P′, 53, on inverted image (of letter “A”) at image plane 54, we see that the route PAP′, ray 55, traverses the edge of lens 56, which is thinner than its center; similarly, PBP′, ray 57, traverses a thicker part of the lens, which slows down the light proportionally, etc. At lens 56 the wave front data exist as an optical planar phase map; the lens's geometric curvatures refracts the rays, or in other words, acts on this phase data ensuring that the time taken for the electromagnetic waves traveling by any route is the same for any given route and pair of points, rays PAP′, PBP′, PCP′, etc. The superposition of waves where their phase coincide—i.e., the simultaneous arrival of wave fronts delineating a point from the image onto the image plane—is the convolution of the interference maxima reconstructing the image of the scene on the image plane (Lipson, op. cit., sect. 4.6, “Convolution,” esp. FIG. 4.11 on p. 92).
Where a glass lens inherently does phase adjustment for convolution, embodiments in this present “Software Defined Lensing” (SDL) invention use computational apparatus and software methodology for phase detection for convolution, working directly on the phase map without a lens. As is known in the art, the Inverse Fourier Transform (i.e., the inverse of a discrete Fourier transform—IFT) expresses a frequency domain function in the time domain, with each value expressed as a complex amplitude that can be interpreted as a magnitude and a phase vector component, as defined for this invention. IFT describes mathematically how a lens convolves interference maxima to reconstruct phase maps into an image, as represented by the phase map's embedded complex amplitude data. In essence, a glass lens refracting light to produce an image is a mechanical analog apparatus “computing” the interference maxima via its curvatures and material properties necessary to form the image through convolution at the lens' focal point. However, in contrast to conventional imaging, using a lens and its sensor where image phase vector convolution functions are fixed (expressed as a point spread or Fraunhofer function), in the present invention embodiments for convolution may apply programmed phase convolution tolerances, discussed further below in the embodiments illustrated in FIGS. 9C, 10C, 13A and 13B.
Multiple wave fronts reflected or transmitted from an object (or scene), at each frequency or wavelength of interest, combine to form planar wave fronts (i.e., wavelets). As known in the art, there are two mathematical models that can be used for analysis: 1) the waves can be described by the Huygens approach (as in FIG. 1) as point sources spreading in circular waves; or, 2) more appropriate for embodiments in this invention, the waves are best described as Fourier plane waves, which facilitate signal analysis of the phase map. This is explained in the following extract from a text on how the light field from an object is optically transferred and reconstructed at the image plane:
“To get an ideal image in the plane wave model (the Fourier approach), the fields in both the object and image planes must be made up from the same set of component plane waves. Each component plane wave must occur with exactly the same amplitude and relative phase in both object and image fields. Alternatively, if the light field just beyond the object transparency is regarded as being made up of many point sources (the Huygens approach), then the light from each point source must be reproduced at the corresponding point in the image plane. It must be reproduced as a point and not spread over a larger area, and it must have the correct amplitude and phase relative to all the other points. [emphasis added] . . . .
“In both models, the light field in the image plane is identical to the light field in the object plane. The field beyond the image is identical to the field beyond the object and therefore the two cannot be distinguished.” (Underlined emphasis in original: The Transfer of Optical Information, Milton Keynes: The Open University, 1995, Units 8 & 9, p. 5, sect. 1.2, ISBN: 0-7492-5160-3.)
FIGS. 2A & 2B (prior art) illustrate the two methods in these models: “[FIG. 2A] The Fourier model (plane wave model) showing plane waves 40 diverging from the object plane 41 and converging at the image plane 42; [FIG. 2B] The Huygens model (point source model) showing spherical waves 43 diverging from points 44 in the object plane 45 and converging at points 46 in the image plane 47” (ibid., p. 5, quoted from caption for FIG. 1 overlaid herein on FIGS. 2A & 2B, bracketed figure and item numbering added). Waves 40 and 43 represent the light fields, illustrated subsequently in FIG. 3.
As known in the art, these wave fronts from the object or scene are represented as a phase map, as illustrated in FIG. 1 at lens 56, which are intercepted between the object plane and the image plane (or virtual image plane); this phase map contains the light field phase vector interference maxima signals conventionally reconstructed by a lens as an image at the image plane.
In the known art of holography, the phase map is captured on film and its reconstructed image is viewed by transmitting a coherent light of the same phase distribution as the reference beam used to make the hologram. The present invention is an electronic and software analogue to a hologram's phase modulation mechanism, i.e. its phase map (Yaroslaysky, L, Digital Holography and Digital Image Processing, Kluwer, 2004, ch. 2.3.2 “Imaging in Fourier Domain: holography and diffraction integrals,” esp. FIGS. 2.9 & 2.10, ISBN: 1-4020-7634-7). In other light field cameras, such as the Lytro, the photon sensors only detect intensity, not phase, using micro lenses for “convolution” following conventional optical methods (U.S. Pat. Nos. 7,723,662 and 7,936,392; further explanations of light field prior art: Levoy, M, “Light fields and computational imaging,” Computer, 2006, v.39, p. 46; Gershun, A, “The Light Field,” Moscow, 1936, trans. by Moon, P & Timoshenko, G, J. Math. and Physics, 1939, v. 18, pp. 51-151).